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ABSTRACT 



Six separate studies were conducted using judgments from academic 
and practitioner experts to determine the structure of the 
Certified Financial Planner (CFP) Professional Education Program 
knowledge (i.e., the organization of discipline topics and the 
relationships that oonsaunicate and establish understanding among 
the topics). The knowledge structure investigations were based on 
meaningful cognitive learning theory where findings were intended 
to assist in the development of professional education 
instructional materials that could help learners (a) solve 
efficiently domain-specific problems, (b) transfer relevant 
knowledge to novel situations within a discipline amd (c) perform 
successfully on certification examinations that aim to assess 
complex thinking behaviors. 

Across the CFP I-VI knowledge domains, the resulting six knowledge 
structures organized an initial 175 discipline topics into 50 key 
areas overall, based on 15,503 degree-of-content-relationship 
values provided by 37 experts for 2,511 topic-pair combinations. 
The data were analyzed for each CFP knowledge domain using the 
principal components analysis technique where relationship 
Judgments were transformed into factor loadings (i.e., topics 
judged to be highly-related were grouped together statistically so 
that a relatively small number of factors could be used to 
represent interrelationships among domain-specific topics). The 
factors that emerged for each CFP knowledge domain were 
interpreted and labeled in reference to the key ideas conveyed by 
the topics that indicated a common underlying dimension. 

For each CFP knowledge domain, findings were presented in a matrix 
that depicted the organization of key ideas and the topics that 
formed them. The resulting matrices produced detailed blueprints 
that could be used to (a) identify and emphasize the key ideas 
within each CFP domain of knowledge, (t) determine and communicate 
relationships among topics to help learners make sense of and 
assign accurate meaning to new information, (c) specify linkages 
among topics to identify material that could be drawn together 
throughout instruction to help learners relate new information to 
prior knowledge in memory and (d) assist in the design of a 
comprehensive review course. 

In sunsnary, the synthesis of both quantitative and qualitative 
knowledge structure assessment procedures produced a useful 
strategy that could offer learning-directed and practical guidance 
for course development efforts related to education in the 
professions. Specifically, the systematic approach applied in 
each of the six investigations offered direction in identifying, 
coordinating and sequencing domain-specific ideas and topics 
within a profession to produce Instructional tasks that could (a) 
make sense to the learner and (b) be related by tne learner to 
relevant, previously acquired information. 



INTRODUCTION 



The College for Financial Planning is recognized nationally as the 
leading provider of distance-based, financial services 
professional education. The Collegers Certified Financial Planner 
(CFP) Professional Education Program, registered with and approved 
by the International Board of Standards and Practices for 
Certified Financial Planners (IBCFP), consists of six sequential 
financial planning knowledge domains j CFP I - Fundamentals of 
Financial Planning, CFP II - Insurance Planning, CFP III 
Investment Planning, CFP IV - Income Tax Planning, CFP V - 
Retirement Planning and Employee Benefits and CFP VI - Estate 
Planning • 

For the past 18 years, the College's home-study materials have 
been designed to help adult learners attain educational goals 
through the use of domain-referenced, measurable objectives (e*g#, 
see models developed by Bloom, Englehart, Furst, Hill & Krathwohl, 
1956; Mager, 1962; Popham, 1981). Course goals and objectives 
have been determined by advisory panels of experts and Job 
analysis studies that have Identified the professional 
requirements needed for competent performance of financial 
planning services* 

THEORETICAL FRAMEWORK 

In keeping informed of cognitive-based conceptions about human 
learning (i.e., the science of understanding the mental activities 
that occur throughout the learning process )y the College for 



« t 

Financial Planning has continued to investigate ways 
to design Instructional materials that will lead novices 
efficiently and effectively toward financial planning expertise • 
Given contemporary educational psychology views In which learning 
is believed to be an active ^ constructive and goal-oriented 
process dependent upon tne mental activities of the learner 
(Anderson, 1985} Ausubel, Novak & Haneslan, 1978; Masters & 
Mislevy, 1990; Shuell, 1986; Van Rossum & Schenk, 198U; West & 
Pines, 1985), the Collegers course development efforts have 
addressed the promotion of meaningful knowledge acquisition. 
Based on principles within the meaningful cognitive learning 
paradigm (see Shuell^ 1990), the structure of financial planning 
knowledge domains is viewed as critical in guiding efforts to 
develop instructional materials that will foster meaningful 
learning. 

OBJECTIVES 

Six separate studies were conducted to identify and interpret the 
structure of knowledge for each component (i.e., knowledge domain) 
of the College's CFP Professional Educational Program. The two 
primi^ry objectives of each study were to determine both the 
organization of and the types of relationships among key topics 
for the six CFP knowledge domains. Both types of information were 
deemed essential in providing guidance for the design of CFP 
instructional materials that would encourage learners to (a) solve 
efficiently financial planning problems, (b) transfer relevant 
knowledge to novel financial planning situations and (c) perform 
successfully on measures that aim to assess hi^er-order thinking 
behavior. y 
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METHODOLOGY 



The IBCFP recently endorsed a total of 175 key topics to represent 
collectively the CFP I-VI knowledge domains, based on six separate 
topic specif lofltion workshops conducted in collaboration with the 
College for Financial Planning during 1989 (see IBCFP, 1990j 
Skumik, 1990). Using the IBCFP key topics for each CFP knowledge 
domain, both quantitative (Diekhoff & Diekhoff, 1982) and 
qualitative (Donald, I983) structural knowledge assessment 
proced;<,res were synthesized to produce a practical technique for 
(a) inferring the organization of professional education knowledge 
domains and (b) communicating topic interrelationships to promote 
meaningful cognitive learning. 

A pilot study of the technique and the data collection instruments 
was conducted during May, 1990, with 16 Academics from the 
College's Education Division. The preliminary research (a) 
verified the utility of the technique, (b) enabled considerations 
about statistical assumptions to be examined regarding the 
instrumentation and the treatment of the data and (c) provided 
insight for the six separate studies that followed (see McCallin & 
Gibley, 1990). 

Sample 

The samples used in each of the six investigations were comprised 
of nationally-recognized experts, including the College's 
content-specialist course developers (total n=39, average sample 
size per CFP knowledge domain =6). The nonrandora samples were 
chosen deliberately to ensure that the detailed data required to 
ascertain expert-based knowledge structures were collected from 



only those Individuals who (a) represented proportionately both 
academic and practitioner financial planning subgroups and (b) 
also wore qualified and willing to provide the in-depth input 
needed. Additional Justification for the sample sizes and 
sampling considerations used within the present studies was based 
on prior research which found that experts generally provided 
similar and very consistent judgments about subject-matter 
I'elationships within their domain of expertise (Chi, Feltovich & 
Glaser, 1981; Chase & Simon, 1973; Larkin, McDermott, Simon & 
Simon, 1980; McKeithen, Reitman, Rueter & Hirtle, 1981; Shavelson, 
197U). 

Research Design 

The descriptive research design was utilized in the six knowledge 
structure studies because the overall goal was aimed at describing 
both the organization of the IBCFP topics and the types of 
relationships among the topics within each CFP knowledge domain. 
The data were collected using a survey approach to obtain experts' 
time-bound associations (i.e., Judgments indicating 
degree-of-oontent-relationship) between all possible 
randomly-ordered, topic-pair combinations within each CFP 
knowledge domain. The invest igaulons were planned to provide a 
descriptive account of each CFP knowledge domain that could be 
uaed to assist in the design of instructional materials that would 
foster meaningful learning. 

Quantitative Measures 
The quantitative data collection instruments were developed to 
obtain values indicating the degree-of-content-relatedness between 
topics for all topic-pair combinations by CFP knowledge domain. 



The measurement approach was based on a numerical relationship 
Judgment technique devised to help Instructors organize and 
communicate structural knowledge within a course or discipline 
area (see Dlekhoff 4 Dlekhoff, 1982). While a number of methods 
for assessing structural knowledge have been proposed over the 
past two decades (e«g«, compare^and-contrast essay tests, concept 
similarity Judgments, card sorting, cued ai!d free recall tasks, 
word associations, graph construction and concept mapping 
techniques) (Biglan, 1973; Champagne, Klopfer, DeSena, & Squires, 
1981; Fenker, 1975; Gorodetsky & Hoz, 1980; Johnson, 1967; 
Naveh -Benjamin, McKeachie, Lin, i Tucker, 1986; Novak, Gowin, & 
Johansen, 1983; Preece, 1976; Shavelson, 1974; West & Pines, 
1985), the approaches generally have been very subjective and time 
intensive to construct and score. 

The numerical relationship Judgment technique proposed by 
Dlekhoff & Dlekhoff offered an alternative to the time-consuming 
and oftentimes cumbersome procedures associated with (a) 
multidimensional scaling analysis of concept similarity Judgments, 
card sorting and free recall tasks, (b) hierarchical cluster 
analysis of word associations, card sorting and graph construction 
tasks, (c) lattice-based ordered tree analysis on cued and free 
recall strings and (d) content analyses on tree construction and 
concept mapping outcomes. In addition to the«e considerations, 
research using simple and multiple correlations between test 
scores (i.e,, a multiple-choice test over knowledge of domain 
topics, an essay test covering relationships among domain topics^ 
and a relationship Judgments test that required rating the 
strength of content relationship between topics presented In 



pairs) provided evidence to support the use of numerical 
relationship judgments as a valid means of assessing domain 
knowledge structure (see Diekhoff, 1983). 

Although numerical relationship judgment measures are not without 
shortcomings (e.g., if psychological processes produce 
asynmotrical distances, it may be difficult to interpret the data; 
usually there are no provisions for obtaining qualitative 
information about prepositional meanings), considerable efforts 
over the past two decades to assess the knowledge structure 
construct indicate that no single technique can be used to provide 
inclusive information. It is for this reason that both 
quantitative and qualitative approaches have been pursued to 
assess multiple aspects of CFP domain knowledge structure. 

The quantitative measures used in the present investigations were 
designed with reference to psychometric considerations not 
addressed in earlier studies (e.g., Biglan, 1973: Diekhoff 1983; 
Diekhoff & Diekhoff, 1982; Wainer & Kaye, 1973). In addition to a 
provision for obtaining qualitative information about topic 
relationships, attempts were made to control for nonrandom 
Judgment error, a reduction in the richness of the raw data and 
violations to assumptions about both the distributions and the 
properties of the aata as well as the Subject pools. 
Specifically, the successive-interval contim um on which 
topic-pair-relatedness values were based involved the assumptions 
that (a) data were dispersed across all units along the defined 
continuum (b) participants were capable of assigning values that 
corresponded to the degree to which stimuli were related within 



their domain-specific cognitive structures and (c) experts within 
each study were similar with respect to domain knowledge, so that 
topic-pair values differed primarily because of unrelated random 
errors in judgment. 

The measurement operation required individuals within each sample 
to Judge independently CFP topic-pair stimuli along a defined, 
degree-of-content-relationship continuum that contained 
successive, equidistant units labeled from one to nine. The scale 
was anchored so that a value of "1*^ indicated that topic- pair 
members were not related beyond a superficial domain association 
whereas a value of "9" conveyed a very strong degree-of-content 
relationship between two topics. The middle interval along the 
continuum (i.e., a value of "^5") designated a moderate 
degree-of -content relationship. In essence, the continuum 
represented a hypothetical range of values from one to nine that 
could be used to express relatedness between two topics within a 
professional education knowledge domain. 

The unique approach used to index topic relatedness was undertaken 
with the recognition that (a) orderinf was the appropriate 
mathematical property applicable to the obtained values, (b) the 
use of the numeric values that approached interval-level 
measurement did not imply the property of uniform distance between 
Vc.lues and (c) the scale limits truncated continuum values (i^e., 
since all topics within each CFP part were related by virtue of 
domain membership, yet none were identical to one another, values 
below one or beyond nine would not have been meaningful). Efforts 
to attain equality of psychological intervals among continuum 
units included the attachment of relatedness values to all 
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intervals. No assumptions were made, however, regarding the 

* 

psychological equality of the nine successive units. It was 
assumed only that the ordered ccxitinuum defined the set of 
possible values that could be assigned during the iwasurement 
process and that unit boundary lines were stable except for 
sampling errors • In addition, a measurement-independent position 
was taken (Burke, 1963) in that c»ce the measuren>ents were 
obtained, they were viewed as numbers (regardless of the original 
scale) that could be subjected to computations and statistical 
analyses as long as the data met assumptions required by the 
methods used (see Gardner, 1975; Ware & Benson, 1975) • 

Efforts to reduce sampling error associai^ed with the pairing and 
proximity error associateu with the presentation of topic-pair 
stimuli were undertaken utilizing a custom-designed computer 
algorithm (Raju, 1990), The program generated all possible random 
combinations of topic-pair stimuli for each CFP knowledge domain. 
Next, the nine-unit, degree-of-content-relationship continuum 
was included at the top of every page containing topic-pair 
stimuli* In this manner, the resulting measurement instrument for 
each CFP knowledge domain contained all possible pairs of key 
topics in a random-order, successive-intervals format that was the 
Sv -'e for all subjects within each study. 

By using the defined continuum to obtain numerical values that 
conveyed structural knowledge information (i.e-, content- 
relatedness Judgments were used to explore the interrelationships 
among topics within each CFP knowledge dnrain), it was possible to 
(a) determine the distribution of the data for each CFP domain of 



knowledge, (b) Identify nonrandom Judgment errors and (c) reduce 
random judgment error by averaging topic-pair values across 
subjects # 

In sunnaarizing, there wre three Important distinctions that 
needed to be addressed prior to concluding discussion about the 
development and the utilization of the data collection 
Instruments . 

First, the criterion upon which topic-pair stimuli were Judged 
was specified as "CFP domain relatedness** (i.e., the degree or 
magnitude to which topics were connected or associated). The 
criterion was viewed as distinct from Judgments based on 
''similarity^ or ''difference** reference frames where topics would 
have been be compared and rated in terms of how identical or how 
different they were from each other. 

Next, the current studies were aimed at determining the degree-of- 
content-relatedness between topics presented in pairs, not whether 
one topic within a pair was of greater or lesser quantity than the 
other in some defined respect. If the latter had been the 
research focus, then the traditional pair comparisons method would 
have been used to obtain and scale the data (i.e., a discriminant 
model could have been ? ,lied to scale ordinal estimates about 
whether one topic within a pair was greater or lesser tnan the 
other topic with raspect to an attribute) (see Thurstone, 1927). 
Given the present research foci, the successive-interval, numbered 
continuum used to Judge the content relatedness of topic-pair 
stimuli served as a valli means through which data relevant to the 
purposes of each investigation could be acquired. 
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Fli Sf the measures wtre developed to obtain experts ♦ Judgments 
in reference to a single, specified attribute (called CFP domain 
relatedness ) > baaed on the overall goals to organize and integrate 
domain knowledge in ways optimal for meaningful learn" g. While 
multidimensional scaling analysis could have been used to 
determine (a) unknown dimensions involved in the Judging of 
topic-pair stimuli or (b) how experts may have used certain 
dimensions when determining topic-pair relatedness^ the primary 
objectives of the current Investigations would not have been 

addressed. 

Procedure 

For each of the six studies ^ experts were contacted by telephone 
to request their participation in an on-site t one-day advisory 
committee session and to confirm their willingness to serve as 
topic-pair Judges prior to the meeting. Following the precontact 
activity, participants in each group received the numerical 
relationship measure containing all possible pairwise 
combinations of topics for their CFP domain of expertise, 
accompanied by (a) a cover letter that outlined the 
responsibilities of respondents, (b) a suimnary of the research 
project, (c) the IBCFP Topic/Subtoplc list for the relevant CFP 
knowledge domain and (d) a OTe-page set of directions that 
specified how stimuli were to be Judged. The topic-pair measure 
for each CFP knowledge domain was separated into manageable parts 
so that experts could Judge each section in about one-hour 
intervals. All participants were contacted approximately two days 
after they received the materials to ensure that everyone 
understood what was required of them and to emphasize the deadline 



for receipt of the data. Data were needed prior to the advisory 
consnittee meetings so that nonrandom Judgments bwovs could be 
reconciled during each on-site session. 



In all six Investigations, respondents wre asked to judge 
independently and individually each pair along the nine-point 
continuiun in terms of the degree-of-content-relatlonship they 
perceived to exist between the topics. A value of "1" was to 
indicate that topics were not related beyond a superficial domain 
association, whereas a value of "^9^ was to convey a very strong 
content relationship^ between topics. Experts also were requested 
to (a) refer frequently to their IBCFP Topic/Subtopic list 
throughout the judgment activity to keep in mind a common context 
and to maintain among participants a consistent interpretation of 
topic labels, (b) consider carefully and repeatedly the 
utilization of all nine units along the continuum when judging 
stimuli and (c) judge the separate topic-pair sections at 
different times to control for rater fatigue. 



Treatment of the Quantitative Data 
Ninety-five percent of the total number of participants across the 
six investigations returned completed topic-pair surveys by the 
specified deadlines. A summary of the data received is provided 
in Table 1. 
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Table 1. Toplo-Palr Data by CFP Knowledge Dooalo 

Resulting # 

# of Key # of All Original & it) of Total # 



IBCFP Pairwise # of Experts Those Who of Topic- 

CFP Domain Topics Topic Combinations Surveyed Responde : Pair Values* 

I 21 210 6 6 (100$) 1,260 

II 31 465 6 5 (83«) 2,325 

III 33 528 7 6 (86J) 3,168 

IV 32 496 7 7 (10016) 3,472 

V 29 406 6 6 (100$) 2,436 

VI 29 406 7 7 (100$) 2,842 

TOTAL 175 2,511 39 37 (95$) 15,503 



•Less than one-tenth of one percent of returned surveys c<Mitained missing values. 
In such instances, missing data were replaced with the average value for the 
observation. 

Examination of Assumptions 

Descriptive analyses were performed on the topic-pair data for 
each CFP knowledge domain. The data distributions were examined 
with reference to the assumptions that (a) the ordered continuum 
defined the set of possible values that could be assigned during 
the measurement process and (b) the topic-pair-relatedness 
Judgments would be dispersed across all units along the defined 
continuum. 
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Histograms of the fraquencles of topic-pair values and smmnary 
statistics were produced by CFP knowledge domain to display and 
describe the shape of data distributions ^ the variability aaw^ng 
relatedness judgments and where typical values were concentrated^ 
The summary statistics appropriate for ordinal-level measurement 
indicated that the distribution of data for each investigation 
satisfied both assumptions as topic-pair-relatedness Judgments 
were spread across all possible values along the continuum (Table 
2). 

Table 2. Si^sary Statistics for CFP I-VI Data Dlstrlbutlims 



CFP 

Domain 


Median 
Value 




Range 


S Newness 




Kurtosls 


I 


U.OOO 




8.000 


.255 




-1.237 


II 


3.000 




8.000 


.575 




-1.039 


III 


5.000 




8.000 


.062 




-1-285 


IV 


2.000 




8.000 


.893 




-.735 


V 


5.000 




8.000 


.107 




-1 .561 


VT 


5.000 




8.000 


-.113 




-1 .207 


Median 


4.500 




8.000 


.181 




-1.222 


While none 


of the six distributions was precisely symmetrical (a 


condition 


seldom met 


and not mandated 


for the analyses 


performed 


on the CFP 


knowledge 


structure data), 


it was clear 


that 


raters in 


each study 


took seriously 


the tasks to 


consider all 


intervals 


along the 


continuum 


and to 


use their discriminative 


powers to 



Judge independently the content relatedness of topic-pair stimuli 
(Table 3). 
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Table 3. Peroant of Data Ai^>earlng within Eac^ 
Ihiit Along the Belatadsess Contlnuua 
by CFP Knowledge DonalB 

Continuum Intervals 



CFP 1 
Domain 1 


One 1 
i%) 1 


Two 
i%) 


j 


Three 1 

i%) I 


Four ! 
($) I 


Five 
i%) 


1 Six 

1 i%) 


1 Seven 

1 it) 


1 Eight 1 Nine 
1 i%) 1 i%) 


1 TOTAL 
1 {%) 


T 1 




14 6 


1 




9.3 1 


10.5 


1 10.2 


1 10,8 


6.9 


1 8.5 


1 100.0 


II 1 


27.3 


17.4 


1 


10.1 1 


8.9 1 


7.6 


1 6.6 


1 8.1 


1 7.6 


1 6.4 


I 100.0 


III 1 


11.8 


15.3 




11.9 1 


8.7 1 


12.4 


1 9.4 


1 14.3 


1 11.8 


1 4.3 


1 100.0 


IV 1 


47.5 


10.4 




7.8 1 


4.3 1 


5.1 


1 4.4 


1 7.5 


1 5.8 


1 7.2 


1 100.0 


V 1 


28.4 


7.5 




7.8 1 


5.3 1 


8.1 


1 5.7 


1 11.0 


1 7.7 


1 18.7 


1 100.0 


VI 1 


7.7 


1 12.5 




11.5 1 


7.3 1 


13.4 


I 10.7 


1 15.5 


1 11.7 


1 9.7 


1 100.0 


Overall 1 


24. 6 


1 12.7 


1 


9.8 I 


7.0 1 


9.4 


1 7.6 


1 11.3 


1 8.7 


8.9 


1 100. 0 



other evidence about the quality of toplc-palr-relatedness 
Judgments was reviewed through efforts to Identify nonrandom 
judgment errcs. The standard deviation of the mean value for 
each topic pair was Inspected to gauge the variability of 
relatedness judgments. It was hypothesized, on an a priori basis, 
that higl",ly variab' 3 topic-pair values generally would indicate 
either varying Interpretations of topic labels (i.e., the primary 
reason for nonrandom judgment errors) or differences in opinions 
about the content relatedness of some topics. Given the small 
sample sizes, the large number of topic pairs within each study 
and past research which indicated that even experts varied in some 
Judgir/^nts across all topic pairs (Diekhoff, 1983; Fenker, 1975), 
it was decided that stimuli which had standard deviations in 
relatedness judgments approaching or exceeding 3*0 would be 
reexamined during the on-site advisory meetings. The standard 
deviation criterion value \ras established by (a) examining the 
range and distribution of variabilities obtained overall 
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(i.e., 0.00 to U.12), (b) considering assumptions about the 
nine-point contlnutun and (c) evaluating the index of dispersion at 
which relatedness Judgments could be considered unequivocally as 
"highly variable". Across all CFP knowledge domains, 3% to 16$ of 
the original judgments required reexamination. 

The reassessment activity was conducted at the beginning of each 
advisory session. Since the intent was to reconcile only 
nonrandom Judgment errors attributable to varying interpretations 
of topic labels (i.e., no attempts were made to reconcile 
idiosyncratic opinions about certain topics), experts revisited 
and discussed the relevant IBCFP Topic/Subtopic descriptions. 
Following the activity to eliminate possible ambiguity of topic 
labels, participants re Judged independently topic pairs, presented 
in a descending order from most to least variable, according to 
the same procedure used to obtain the initial relatedness 
Judgments. Subjects were not knowledgable about (a) the manner in 
which stimuli to be re Judged were presented or (b) the original 
values of the topic pairs. In each study, however, experts 
noticed that certain topics appeared frequently In the pairs that 
needed to be reexamined. This was found to be the case especially 
among the topics within stimuli that received relatively 
more-dispersed original content-relatedneas values and hence, were 
listed at the beginning of the rerating instrument. This 
observation supported further the notion that varying 
Interpretations of topic labels contributed significantly to 
nonrandom Judgment errors. 
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Results from the reassessment exerolse showed a marked Improvement 
by reducing the proportion of topic-pair-relatedness values that 
were highly variable (Table 4), Even so, it was recognized and 
expected that experts would vary to some degree in their judgments 
due to both random Judgment error and differing opinions about 
some topics (Biglan, 1973? Chi, et al,, 1981? Diekhoff, 1983; 
Fenker, 1975; Schoenfeld & Herrmann, 1982; West i Pines, 1985). 

Table 9. Status of Highly-Variable Topic-Pair 
Belatedness JudgBmits Before and 
After Berating Ex«?olse 

Status 









Proportion of 










Proportion of 


R era ted 










Berated 


Pairs that 


Proportion of 






Proportion of 


Pairs that 


Remained 


Rerated Pairs 


1 Proportion of 


CFP 1 


Highly-Variable 


Decreased in 


1 Unchanged in 


1 that Increased 


1 Highly-Variable 


Domain 1 


Original Judgnents 


Variability 


1 Variability 


1 in Variability 


1 Final Judgments 




it) 


i%) 


1 i%) 


1 (%) 


1 (*) 


I 1 


13.33 


1 71.43 


1 17.86 


I 10.71 


1 2.86 


II 1 


6.67 


1 65.53 


1 32.24 


1 3.23 


1 0.86 


III 1 


3.03 


1 25 . 00 


1 68.75 


1 6.25 


! 0.57 


IV 1 


10.89 


1 51.86 


1 44.44 


1 3.70 


1 1.61 


V 1 


15.52 


1 49.21 


1 25.40 


1 25.39 


1 7.88 


VI 1 


4.93 


1 45.00 


1 50.00 


1 5.00 


I 0.i9 


Median! 


8.78 


1 50.54 


I 38.34 


1 5.63 


1 1.24 



Following the reratlng activity to minimize nonrandom Judgment 
error, the objectivity of experts' Judgments was examined with 
respect to the assumptions that (a) participants were capable of 
assigning values that corresponded to the degree to which stimuli 
were related within their domain-specific cognitive structures and 



o 
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(b) respondents within each CFP study were similar with respect to 
domain knowledge so that relatedness values differed primarily 
because of unrelated^ random Judgment errors. 

It was hypothesized that if experts were similar in domain 
knowledge and capable also of assigning values that represented 
the structure of that knowledge, then their topic-pair-relatedness 
judgments would be related linearly and higlily. In essence t the 
internal consistency among experts was viewed as an index of both 
judgment reliability and construct validity. 

The interdependent assumptions were tested by computing (a) 
inter-rater correlation coefficients, (b) sunmary and rater-total 
statistics and (c) Cronbach's Alpha reliability coefficients to 
examine the extent of agreement among experts within each 
Investigation. As summarized in Table 5f experts were found to be 
very consistent in their relatedness judgments. 



Table 5. Reliability of CFP I-?I Topic-Pair 
Relatednesa Judgaenta 



CFP Do; lain 



Team 

Reliability 
(Alpha) 

Average 

Inter-Rater 

Correlation 



t I 

1 (n=6) 


II 1 
(n=5) 1 


III 
(n=6) 


IV 1 
(a=7) 1 


V 

(n=6) 


VI 1 
(n=7) 1 


Overall j 
(Median) 1 


1 .8U 


.88 1 


.85 


.90 I 


.90 


.83 1 


.87 1 


1 .«7 


.60 1 


.50 


.58 1 


.62 


.41 t 


.54 1 
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The high Internal consistency coefficients also provided evidence 
about validity of the obtained measurements where the Intent was 
that experts Judged stimuli according to cme dimension of 
psychological variation (l#e#, CFP domain relatedness) • Thus, 
even though the stimuli Judged could have been multidimensional t 
the hi^ internal consistency indices supported the assumption 
that experts generally were successful in Judging topic pairs 
according to the unldimensional, degree-of-content-relatlonship 
continuum* 

The information^ combined with the observation that respondents 
within each study had fairly comparable variances (i^e.f alphas 
and standardized item alphas virtually were identical), furnished 
proof that the values obtained generally were (a) associated 
linearly and (b) unidimensional with reference to the relatedness 
continuum. 

In summary, it was determined that assumptions about the 
data distributions as well as the subject pools were met in all 
six Investigations, Efforts to minimize nonrandom judgment errors 
also were found to be successful. The status of the data was 
regarded as ordinal measurement and only approached that of an 
interval scale. Like any successive-interval data, methods of 
correction and scaling could have been used to transform values 
(more or Irj^s successfully) into interval measurements (Guilford, 
195^). However, such procedures were not deemed necessary because 
data were subjected only to statistical methods where (a) the 
distributional assumptions Justified the analyses and (b) it could 
be demonstrated that usefulness of the values was enhanced by the 
treatment (Crocker & Algina, 1986). 
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Scoring and Analyses 



The raw data representing the csontent relatedness of topic-pair 
stimuli still contained a random error oompaaent that could be 
minimized further. Across all studies, medians were computed from 
the original 15 f 503 topic-pair-relatedness judgments to produce 
2,5n ingle values. The n»dian was chosen as the measure of 
central tendency with reference to the ordinal mathematical 
property of values along the numerical relationship continuum and 
to minimize the influence of the few highly- variable, outlying 
values that remained after the rerating activity. 

Based on the Diekhoffs' structural knowledge assessment method, 
the median of the numerical ratings for each topic pair by CFP 
knowledge domain was transformed into a decimal notation to serve 
as an index of correspondence (i,e., a median relatedness ' alue of 
1 became .1, a 2 became ,2, etc»). In essence, the median 
topic-pair-relatedness values were viewed as quasi-correlations 
that could be used to identify structural interrelationships. 

The pragmatism of the technique was appealing yet also approached 
with caution in that it was imperative to recognize limitations of 
treating median topic-pair-relatedness values as if they were 
correlations. The approach published by the Diekhoffs' failed to 
address assumptions about linear association, the distributions 
and properties of the data, the subject pools and the measurement 
instruments. While all of these concerns have b^n addressed 
already, the following section supplemented further information 
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about Issues what vrore acknowledged prior to treating the decimal 
form of the naedian topic-pair-relatedness values as 
'^correlations'* • 

The caitinuum values used to index the magnitude of content 
relatednesa between topics mre viewed as numerical indices of 
••going togetherness" not unlike iMSsures of relationship 
communicated through correlation coefficients • Such an approach 
has not been unusual In research about the knowledge structure 
construct as other quasi-correlation coefficients also have 
emerged over the past 20 years to communicate aspects of 
relatedness between topics in terms of the distance between their 
semantic profiles (i.e., psychological representations In 
long-term memory). 

For word association data, analyses usually have involved the 
computation of a coefficient where the relatedness between words 
was expressed according to the degree of overlap of their property 
lists (see Garskof & Houston, 1963; Johnson , 1967; Johnson, 1967? 
Shavelson, 1974). The relatedness coefficients have been entered 
into a symmetric matrix and analyzed using factor analysis, 
multidimensional scaling or hierarchical cluster analysis 
techniques to examine the structure underlying the observed 
coefficients • In graph construction approaches, the distance 
between any pair of wrds has been calculated by adding the 
numbers associated with the line or lines connecting the pair of 
words (see Rapoport, 1967; Rapoport & Fillenbaum, 1972; Shavelson, 
1974). The distances between all pairs of words on the graph(s) 
have been represented in either separate symmetric proximity 



matrices for individuals or in a mean or median matrix for a 
group. Cluster analysis or multidimensional scaling techniques 
have been applied to uncover the dimensions underlying the 
proximity data* Other techniques have been based on a clustering 
algorithm (Reitman & Rueter, 1980) where sets of cued and noncued 
recall strings have been transformed into ordered trees. The 
relatedness of two or more trees has been assessed using a 
nonmetric measure of resemblence (see Hirtle, 1982; McKeithen et 
al., 1981 ; Naveh-Benjamin et al., 1986)* 

In summary, nonmetric, quasi-correlational indices have been used 
for a number of years to represent aspects of knowledge 
structures. The data have been analyzed using various statistical 
techniques (i.e., primarily factor analysis, multidimensional 
scaling or hierarchical cluster analysis) where each method has 
involved different assumptions about the nature of the observed 
data and the structure or dimensions underlying them. 

Overall, it was concluded that there was sufficient Justification 
for treating the median topic-pair relatedness values as 
quasi-correlational indices that were (a) not obtained by chance 
alone, (b) indicative of the exfent to which discipline topics 
were associated with respect to the unidimensional attribute **CFP 
domain relatedness" and (c) meaningful in so far as values were 
within the degree-of -content-relationship continuum Intervals of 
one through nine (i.e, all topics by CFP study were related at 
least superficially by virtue of domain membership, yet none of 
the topics had identical, completely overlapping property lists). 
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The topic-pair median indices for each CFF knowledge domain '.vere 
entered into separate symmetrical matrices and treated as though 
they were correlations. Principal components analysis was used to 
group hi^ly-related topics together so that a relatively small 
number of "key idea areas" could be used to represent the 
underlying structure of each CFP knowledge domain (Harman, 1967). 
The appropriateness of utilizing the factor analytic model was 
evaluated using the Kaiser-Meyer-Olkin (KMO) measure of sampling 
adequacy index to compare the magnitudes of the obtained 
topic-pair-relatedness indices to the magnitudes of the partial 
topic-pair-relatedness indices (i.e., the quasl-correlations 
between pairs of topics were examined to determine the degree to 
which they could be explained by the other domain-specific 
topics). As shown in Table 6, the KMO statistic for each CFP 
knowledge domain was high enough to Justify use of the factor 
analytic model (Norusis, 1988). 

Table 6. Kalsez^-MeyeiMMkln M0aauz*e 
of Sampling Adequacy by 
CFP Knowledge Dooain 

CFP Domain KMO Index 

I .770 

II .787 

III .803 

IV .805 

V .930 

VI .852 

Median ^ .80^ 



7 
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Using SPSS/PC^ microcomputer data management and analysis 
software, the principal compaients analysi.o technique was applied 
to each of the six matrices to obtain uncc. rt!lated linear 
combinations for factor extraction (Hotelling 1933) # The 
principal components technique was utilized because it provided an 
appropriate solution for condensing statistically the 
topic-pair-relatedness data into a limited number of 
linearly-independent factors to discover the underlying order of 
each CFP knowledge domain (see Thurstone, 1947 )• 

Three sources of Information were examined to determine the number 
of factors needed to represent parsimoniously the relationships 
among sets of the many interrelated topics within each study. 
First, the total variance explained by each factor was examined, 
accompanied by both the percentage and the cumulative percentage 
of variance attributable to respective factors^ Next, the factors 
that accounted for variances greater than one also were reviewed 
(i.e^, factors with eigenvalues below one generally were no more 
efficient than a single topic was in representing the data). 
Finally, scree plots of the eigenvalues associated with each 
factor were evaluated in light of the preceding information. 
The combined sources indicated the utilization of models ranging 
from 3ix to eleven factors (Table 7). 
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Tabid 7- Simsiary of Factor Extraction Data by 
CFP Knowledge Dosaln 



Factor Statistics 



Factor 



CFP I 
Eigen- Variance 
value Explained 



CFP II 
Eigen- Variance 
value Explained 



CFP III 
Eigen- Variance 
value Explained 



Extracted I 




i%) 




i%) 1 




i%) 




i%) 




($) 1 




($) 1 


1 1 


9.65 


(45.9)1 


12.48 


(40.3)i 


16.45 


(49.8) 


10.86 


(33.9) 


14.65 


(50.5) 


15.69 


(54.1) 1 


2 1 


2.63 


(12.5) 


4.98 


(16.1)1 


2.95 


(8.9) 


3.38 


(10.6) 


4.26 


(14.7)1 


2.94 


(10.2) 1 


3 1 


2.^^ 


(10.5) 1 


3.66 


(11.8) 1 


2.23 


(6.8)^ 


2.40 


(7.5)1 


3.19 


(ll.O)i 


2.43 


(8.4) 1 


1 1 


1.40 


(6.7)1 


2.19 


(7.0)i 


2.06 


(6.2)1 


1.97 


(6.2)1 


1.74 


(6.0)1 


1.65 


(5.7)1 


5 1 


1.31 


(6.2)1 


1.57 


(5.1)1 


1.91 


(5.8) 1 


1 .82 


(5.7)1 


1 .55 


(5.4) 1 


1.41 


(4.9) 1 


6 1 


1.18 


(5.6)1 


1.45 


(4.7)1 


1.61 


(4.9)1 


1 .70 


(5.3)1 


1.43 


(4.9)1 


1.37 


(4.7)1 


7 1 






1.29 


(4.2)1 


1.44 


(4.4)i 


1.46 


(4.6)1 


1.13 


(3.9)1 


1.27 


(4.4)1 


8 1 






1.01 


(3.2)1 


1.31 


(4.0)1 


1.36 


(4.3)1 






1 .05 


(3.6)1 


9 1 










1.14 


(3.5)1 


1 .23 


(3.8)1 










10 1 










1.13 


(3.4)1 


1.15 


(3.6)1 










11 I 














1 .06 


(3.3)1 










Cumulative I 
Variance I 
Explained i 




87.6$ 1 




92.3$ 1 




97.7* 1 




88.7$ i 




96.4$ 1 




95.9$ 1 



CFP IV 
Eigen- Variance 
value Explained 



CFP V 
Eigen- Variance 
value Explained 



CFP VI I 
Eigen- Variance 
value Explained 



'in 
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Following the factor extraction phase, varimax rotation was used 
to transform each of the six matrices to simple structure for 
Interpretation # Allowing for correlations among factors (l.e#, 
achieving simple structure through oblique rotation) did not vary 
significantly from or clarify further any of the six matrices that 
emerged from the orthogonal rotations. Thus, only findings 
obtained from the varimax method were presented. In addition^ 
plots of factor loadings and factor score estimations were deemed 
tangential to the research objectives; therefore, neither were 
reported. 
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Qualitative Assessmdnt 



The content of the factors which emerged from the 
content-relatedness data were reviewed by the Collegers 
subject-matter experts within each CFP knowledge domain* The 
experts Interpreted and named the key idea conveyed by the topics 
which loaded significantly (i.e., .40 or greater) on each factor. 
Considerations involved in the Interpretation and naming of 
factors* key ideas included (a) the sophistication of and 
expectations about learners' domain-relevant prior knowledge, (b) 
how factor labels typically should be understood by learners and 
(c) how factor labels communicated essential aspects of a CFP 
knowledge domain. 

Next, the same Individuals were asked to utilize six 
types-of-toplc-relatlonship categories (Donald, I983) to identify 
the kinds of connections and processes required to (a) comprehend 
the underlying key ideas and (b) determine the ways in which such 
ideas fit into the overall context of a particular CFP knowledge 
domain. The qualitative categories, accompanied by brief 
descriptions, have been listed in Table 8. 



Table 8. Types of Topic Relati<»shl|^ Categorlea 
(Donald p 1983) 



Relationship 
Category 



Description 



Asaoclatlve 



To'^acs define, describe or identify 
each other. 



Funotlooal 



Topics have a similar outcome or 
purpose • 



Structural 



Some of the topics are classes or 
subsets of the other topics • 



Procedural 



Topics are or can be ordered to 
express a known sequence or 
creratlon. 



Logical 



Topics have a logical or 
conditional order • 



Causal 



Topics have an explicit 
cause-effect relationship. 



Finally, the general lessons or study assignments within each CFP 
knowledge domain (i.e., course syllabi were designed with 
reference to advisory committee input and outcomes from the 
knowledge structure research) were linked to the topics they 
addressed \;ithln each factor. Academics utilized the combined 
sources of information to assist them in their CFP professional 
education course development efforts^ 
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RESULTS 



The resulting number of factors by CFP knowledge domain and the 
corresponding percentage of total variance explained (i.e., the 
extent to which factor loadings accounted for relationships among 
topics within each CFP knowledge domain) have been suianarized in 
Table 9. 

Table 9. Hunber of Factors (Key Ideas) and 

Percentage of Total Variance Aoo<Ninted 
for by CFP Knowledge Dosaln 



CFP Domain 


Resulting Number 
of 
Factors 


Percentage of Total 
Variance Explained 
i%) 


I 


6 


(87.6) 


II 


8 


(92.3) 


III 


10 


(97.7) 


IV 


11 


(88.7) 


V 


7 


(96.4) 


VI 


8 


(95.9) 


Median 


8 


(94.1) 



The factor labels assigned by the CDllege's subject-matter experts 
for each CFP knowledge domain are given in Table 10. 
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Table 10. Factor Labels Bei»*«s0tttlng Key Ideas vlthln Each CFP Knovled^ DoBain 



Factor 
« 



II 



HI 



CFP D<Maln 



IV 



?I 



1 i Financial 
I Planning 
I Functions 

I 

I ^ 



Life 

Insurance 



Theoretical 
Portfolio 
C<»struotioa 
& Management 



Pix)perty 
Dispositlcms 



Types of 
Qualified 
Retireoent 
Plans & 

C«isiderations 



Gift Tax 



—1 

2 1 Economic 

|Cc»xoepts 

I 



1 



Medical 
Insurance 



Fixed Income 
Securities 



Direct Partici- 
pati<m Progroms 



Personal 
Contributwy 
Retirement 
Plans 



Gross 

Estate 

Inclusion 



3 I Ethical, 
i Regulatory 
|& Legal 
I Issues 



Property 
& Uability 
Insurance 



Real Estate 
& Tangible 
Assets 



Compwsatico 
Issues & Em- 
ployee Benefits 



Types of 
Employee 
Benefits Plans 
& Cc^sideraticms 



Estate 
Calculaticm 
& Valuation 



1 



"H 

4 I Factors 
1 Affecting 
I Financial 
I Statements 



Disability 
Insurance 



Futures & 
Opticms 



Penalties & 

Additional 

Taxes 



Funding Con-- 
siderati<»s for 
Defined Benefit 
Plans 



Planning 
for 

Incapacity 



1 

ITime 
lvalue 
jof Mcmey 



Lav & 
Industry 



Applied 
Portfolio 
Cmatructicm 
& ManagoMnt 



Tax Aspects 
of Securities 



ManagCTent/ 
Tax Ccmsid«* 
eraticms of 
Contributory 
Funds 



Federal 

Estate 

Tax 

Deduct lOTS 



T 

I Debt and 
I Savings 



Auto/LiabU- 
Ity Insurance 



DiverslTicatiCTi 
& Investment 
Strategies 



Charitable 
Contributions 



Social 

Security 

Consideratlcwis 



Probate 
Considera- 
tions 



7 I 



Social 
Insurance 



Cash Equivalent 
Considerations 



Accounting 
Methods 



Non-- Qualified 
Deferred Com- 
pensation Plans/ 
Considerations 



Liquidity 
Planning 



8 



Related 

Insurance 

Factors 



Regulation of 
Securities & 
Markets 



Social Security 
& Self -Employment 
Tax 



Charitable 
Transfers 



9 I 



Insurance- 
Based 

Investments 



Intrafamily 
Transfers 



10 I 



Economic Consid- 
erations In Port- 
folio Construc- 
tion & Management 



Mfe Insurance 
Ani.uitles 



11 



I Other Tax Issues 
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CFP Profes3ional Education Knowledge Structure Maps 

The detailed outcomes from each investigation were summarized in 
separate charts by CFP knowledge domain. For each matrix, the key 
ideas and their respective topics, accompanied by IBCFP topic 
identification numbers , were listed in the order that they emerged 
after the factor extraction and rotation phases. In addition, the 
CFP study guide assignments, developed by College Academics in 
conjunction with advisory committee input and feedback from the 
knowledge structure research, were listed in parentheses with the 
topics that they covered. 

Due to the proprietary nature of the material contained in the 
resulting matrices, the actual charts produced were not Included 
in this report. Instead, Table 11, designed with reference to 
findings from the CFP I study, was prepared to illustrate 
(Irregardless of domain content) how findings could be utilized in 
the design of professional education Instructional materials that 
could encourage meaningful knowledge acquisition. 

In essence, the following map provides a comprehensive blueprint 
that can be utilized to (a) Identify and emphasize key ideas and 
their components within a professional education knowledge 
domain, (b) determine and conmunicate relationships anong topics 
to help students make sense of and develop accurate conceptions 
about new information and (c) specify linkages among topics to 
help learners relate new information to their prior knowledge and 
practice transferring that knowledge to novel situations. 



Table 11. CFP I Key Areas ^ Toplo Loadings and 
Study Qulde Asalgnment dorrespondeooe 



CFP I I FUHDAMOnrALS OF FUASCIAL PLAHMISa 



Flnanolal 

Planning 

Funotlons 



Eoooaolo 

Ck>O06pt8 



Ethloal, 
Begulatory, 
and Legal Issues 



Factors Affeat4-Qg 
Flnanolal 
Stateaents 



Tiae TaliM 
of ffcMwy 



Debt and 
Savings 



I 
I 



Topic #05 (A1) 



Topic #02 (A13) 



Topic #18 (AID 



Topic #03 (A15) 



Topic #01 <A.) 



Topic #06 (A15) 



Toplo #08 {A8,9»10) 



Toplo #09 (A8,9»10) 



Toplo #04 (A3, 4, 5,6, 7) 



Toplo #07 (A8,9,10) 



Toplo #21 IA2) 



Toplo #20 (A2) 



Toplo #17 (A2) 



Topic #15 (A7) 



Toplo #11 U5) 



Toplo #13 (A13) 



Toplo #11 (All, 12) 



Toplo #12 (A12) 



Seooodary Loadings 



Toplo #40 (A*) 



Toplo #T6 (A2) 



Toplo #19 (AU) 



Seooodary Loading 



Tertiary Loading 



Secondary Loading 



Topic #16 (A2) 



Toplo #19 (AU) 



Toplo #07 (A8,9,10) 



Topic #13 (A13) 



Topic #10 (Ail) 



Seooodary Loading 



Topic #19 (All) 



*(A#) = Study Guide Aaslgnaent Niflaber 
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DISCUSSION 



The numerical relationship Judgment technique led to the 
creation of an objective, expert-based knowledge structure for 
each CFP knowledge domain. In addition, the six 
types-of-topic-relationship categories utilized by College 
Academics provided information about how to develop meaningful 
lessons for the material to be learned in each CFP study guide. 
If relationships among some topics were found to be structural, 
for example, learners could be made aware of the underlying 
superordinate-subordinate hierarchy. Likewise, if topics had a 
procedural association, students could be instructed about the 
order and sequence of steps involved (see Donald, 1983). 

It was recognized that a learner's transition from novice to 
expert status generally involved a progression from simpler to 
more meaningful learning behaviors (Anderson, 1985). Thus, 
findings from the six separate studies were approached with the 
following three ideas in mind; 

• Initially, learners usually acquired domain-specific 
knowledge in a more-or-less rote fashion where isolated 
facts were interpreted according to and subsumed into 
preexisting relevant knowledge bases; 

# Even If the resulting structures and Interrelationships 
among the IBCFP topics made sense to experts, there were 
no guarantees that novices also would comprehend such 
organization and; 



• Learning was not an additive activity but Instead, 

c<xisisted of gradual changes , where relationships among 
acquired factual information enabled the development of 
cognitive conceptual networks that facilitated 
understanding and higher-order problem solving (see 
Shuell, 1990). 

The College's course developers did not view the knowledge 
structure research outcomes in isolation. The findings were 
reviewed in combination with both advisory comnittee input and 
personal professional Judgment to elicit collective guidance in 
study guide development efforts. Thus, the research was rot 
intended to be a panacea for course design methodology* Instead 
the findings presented in the actual knowledge structure charts 
offered concise views of how results from the research could 
assist in integrating the teaching of the IBCFP topics to enhanc 
learner understanding. The ways in which the research outcomes 
could be utilized to bring together and to help make meaningful 
subject-matter content within a professional education knowledge 
domain have been addressed, as follows, with reference to the 
general findings presented in Table 1 1 . 

Domain Knowledge Structure 

The expert-based knowledge structure that emerged from the CFP 
I topic-pair data complemented yet offered a more elaborate 
depiction of the domain (see Table 11), compared to the 
topic/subtopic list published by the IBCFP. 

■if/ 
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The 21 CFP I topics listed by the IBCFP are structured generally 
in either linear sequences or as separate units. Instructional 
materials designed according to knowledge organized in this 
fashion are likely to foster lower-level knowledge acquisition 
(i.e., chaining) because students are given primarily sequential 
and often Isolated domain subject matter that is not linked 
clearly within and across discipline compcMents to encourage 
conceptual knowledge development. 

The arrangement of the topics which comprised factors (i.e., key 
ideas) within the CFP I knowledge structure chart also differed 
sequentially from that published by the IBCFP. As shown in Table 
11, topics formed key ideas on the basis of how experts' 
content-relatedness perceptions revealed unique areas that were 
independent from one another and representative of the CFP I 
knowledge domain. Thene underlying, complex notions about the 
domain were not recognized readily when examining the 
chronological IBCFP topic list. Althougii the organization 
presented in the CFP I knowledge structure map as well as the 
IBCFP CFP I Topic/Subtopic list made sense, a greater integration 
of topics within (i.e., primary factor loadings) and across (i.e., 
secondary and tertiary factor loadings) the key areas for the CFP 
I knowledge domain was made possible through the research 
outcomes. This int<:gration and linkage has been found to be 
related significantly to the development of hi^-quality learning 
outcomes (Van Rossum & Schenk, I98U). 
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Integration within Factors 



The CFP I topic loadings indicated that even though son» topics 
had loadings greater than .40 on more than one factor (i#e#t 
seccaxdary and/or tertiary loadings), topics overall correlated 
highly with only one primary factor. 

In each study, the topics that clustered with one another were 
examined in at least three ways to help course developers be aware 
of the strategies that could help to encourage meaningful 
learning. First, the overall, meaning or underlying theme 
coonaunicated by topics that loaded highly on a factor was 
interpreted by subject matter experts. In most cases, experts 
indicated that factors were interpreted easily. Even so, it was 
not surprising that factors with eigenvalues only sli^tly greater 
than one were found to be more difficult to Interpret by experts^ 
compared to preceding factors that explaiiied relatively higher 
pix)portlons of the total variance. 

The initial activity of Identifying and interpreting key Ideas 
based on topic loadings required experts to consider (a) where and 
how factors and their components fit Into a particular CFP 
knowledge domain, (b) how novice learners might perceive factor 
labels and (c) expectations about learners* preexisting 
domain- specific knowledge. In addition, the relationship 
categorization exercise was intended to make Academics conscious 
of (a) topic interrelationships often taken for granted by experts 
and (b) the various learning strategies that could be consldereo 
to help learners make sense of and relate topics to support 
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conceptual knowledge development. Considerable dialogue ensued 
between the primary investigator, trained in educational 
psychology, and each Academic expert within a particular CFP 
knowledge domain. Specifically, ideas were discussed about 
human- Information processing, meaningful cognitive learning and 
the transfer of knowledge to novel problem-solving tasks. 

Finally, the study guide assignment numbers accompanying the 
tjpics that clustered together, as shown in Table 11, also 
provided clues about how lessons could be linked to foster quality 
learning and transfer of domain content. Topics that formed key 
ideas, yet were covered across various study guide assignments, 
could be connected by Interweaving what was taught in the earlier 
assignments with mat<?rial covered in subsequent lessons. For 
example, in Table 11, the first factor (Financial Planning 
Functions) indicated that two topics covered in the CFP I Study 
Guide Assignment 1 (i.e., Topic #1 and Topic #5) could be 
revisited and/or connected to other topics that loaded on the same 
factor but were addressed in Assignments 13f 1^ and 15 (i#e#. 
Topic #2, Topic *18, Topic #19, Topic #3 and Topic #6, 
respectively . Likewise, the second factor in Table 11, ^Economic 
Concepts'*, was comprised primarily of IBCFP topics covered acoss 
the CFP I Study Guide Assignments 8, 9 and 10. However, Topic #4 
and Topic #10 (Assignments 3 through 7) also were tied into the 
Economic Concepts factor. Thus, facts and principles presented in 
the preceding ass5gnments (i.e», lessons 4, 5, 6, and 7) could 
be linked to the new information given in Assignments 8, 9 and 10 
to enhance learner comprehension and encourage knowledge transfer. 
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In this manner, the topics that formed kev Ideas for the CFP I 
knowledge domain could be connected meaningfully within a factor 
by considering both (a) the accompanying study guide assignments 
that indicated when and where topics were addressed and (b) the 
types of relationships and learning strategies indicated in the 
preceding categorization exercise. The information, combined with 
the meaning conanunicated by each key idea that emerged from topic 
loadings, could be used to help establish additional connections 
in support of meaningful knowledge acquisition and the development 
of expert-like cognitive structures for learners within the CFP I 
knowledge domain. 

Integration across Factors 

The information presented In Table 11 could be used also to 
identify (a) topics that loaded significantly on more than one 
factor (i.e., secondary and tertiary loadings) and (b) the 
structure and components of the CFP I portion of a comprehensive 
review course. 

In the first case, course developers initially had to determine 
the plausibility of secondary and tertiary loadings. Since 
rotation to simple structure was achieved more-or-less 
successfully In the six studies, topics usually loaded highly on 
only one factor. The only exception involved three CFP topics 
where primary loadings were marginal at best* In every study, 
however, lower yet significant (e.g., .40 - .60) secondary and 
tertiary topic loadings also were observed. 
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CFP Academics reviewed the secondary and tertiary topic loadings 
and In most cases, Juiged them to have less- important yet valid 
ccHineotions with the other topics that defined the key idea 
underlying the factor. 

Topics that had significant and valid loadings on more than one 
factor could be presented in course materials in the various ways 
that they mig^t be encountered within a domain. For example , 
Topic #19 was covered specifically in the CFP I Study Guide 
Assignment 14. Yet^ Topic #19 loaded significantly on three 
factors as shown in Table 11 (e.g., primary loading • "Debt and 
Savings", secondary loading - "Financial Planning Functions", 
tertiary loading - "Factors Affecting Financial Statements"). 
Although the topic was to be covered in only one study guide 
lesson, experts* Judgments indicated three key idea areas where 
the topic was prevalent. Knowing how the topic functioned across 
the three factors, as determined by the kinds of relationships it 
had with the other topics that clustered on each factor, could be 
useful in helping learners establish complex links across areas 
within a domain. 

The utilization of secondary and tertiary topic loadings needs to 
be Judged with reference to (a) what would be involved in making 
the best use of available information and (b) the potential 
benefits that could result from efforts to establish complex views 
about a discipline. In most instances, it seems desirable that 
course developers have as many resources as possible to assist 
them in their efforts to design instructional materials that can 
help learners acquire domain-specific expertise. 




Findings from the CFP I knowledge structure study also could be 
utilized in the design of a comprehensive CFP I-VI review course • 
Overall, the six resulting expert-based knowledge structures 
organized the 175 IBCFP topics into 50 key areas (see Table 10). 
The 50 key ajreas accompanied by (a) their respective topics, (b) 
Information about how topics were related both vertically (within 
a factor) and horizontally (across factors) and (c) study guide 
assignment numbers, supplied an excellent framework for the design 
of a capstone course. Furthermore, students at the review course 
stage in the Collegers CFP Professional Education Program should 
be capable of comprehending more like experts, similarities and 
relationships among topics across the six knowledge domains. For 
instance, what previously might have been conceptual yet very 
specific ideas contained within the CFP I knowledge domain could 
be Integrated, on a larger-scale, by using the comprehensive 
blueprint shown in Table 11 in conjunction with the other 
knowledge structure matrices obtained for the CFP II-VI domains. 

In summary, each matrix provided concise, expert views about (a) 
predominant key idea au^eas within the Collegers CFP Professional 
Education Program, (b) the organization of financial planning 
ideas and topics and (c) the coordination and sequencing of 
discipline content. This information could provide the guidance 
needed to design a comprehensive review course where the content 
of each CFP knowledge domain and accompanying learning objectives 
could be Si^ructured to enhance the retention and transfer of 
relevant knowledge required for successful financial planning 
problem solving. 



CONCLUSION 



The determination of the structure of knowledge for each CFP 
knowledge domain was undertaken to provide one source of guidance 
in the design of the Collegers CFP Professional Education study 
guides* Baaed on meaningful cognitive learning theory, the 
research was conducted with reference to the widely-supported 
premise that high-quality learning Involves an active, 
constructive and goal-oriented process in conjunction with a 
learner *s inclination to make sense of and relate new information 
to relevant prior knowledge in memory (see Shuell, 1990). This 
notion, combined with over two decades of educational research 
findings that have confirmed collectively the value of 
understanding rather than memorizing content of a domain, 
supported pursuit of the six CFP knowledge structure 
investigations* 

Quantitative and qualitative structural knowledge assessment 
procedures were synthesized to produce a systematic, practical 
technique for (a) Inferring the organization of knowledge (i.e., 
the IBCFP topics) for each component of the College's CFP 
Professional Education Program and (b) conmunicating 
interrelationships among the IBCFP topics to promote conceptual 
knowledge development. Both types of information were deemed 
essential for the design of instructional materials that would 
encourage successful financial planning problem solving, knowledge 
transfer and favorable performance on examinations aimed at 
assessing higher-order thinking behavior. 
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Six separate studies verified both the validity and the utility of 
the procedure in fulfilling the primary objeotives to determine 
the organization of as well as the relationships among the IBCFP 
topics within each CFP knowledge domain. It was recognized, at 
the onset, that the Collegers distance-based model of instruction 
limited opportunities to ascertain learner disposition and 
motivation. The research confirmed, however, that the structures 
and strategies identified to help students (a) make sense of the 
IBCFP topics and (b) relate the IBCFP topics to promote ccaceptual 
understanding of the financial planning discipline, generally were 
very consistent with currrent course development practices at the 
College. This feedback, combined with the finding that both 
in-house and outside experts' judgments converged consistently, 
provided evidence that supported further the existing credibility 
earned by the College's materials. 

In summary, the detailed outcomes provide a comprehensive 
framework that Academics can continue to utilize and build upon in 
both the development of CFP study guides and the design of a CFP 
I-VI review course. 
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